Perspective matters: assessment of medical students' communication and interpersonal skills by simulated patients from the internal and external patient perspective

Background: Communication and interpersonals skills are important qualities of professionalism in medicine. In medical curricula, they are usually acquired in communication trainings and assessed in OSCEs. Studies show correlations as well as differences between communication ratings of examiners and simulated patients. In our study, simulated patients assessed medical students’ communication and interpersonal skills after a consultation hour from the internal and the external patient perspective. Methods: In December 2019, 52 final-year medical students participated in a consultation hour as part of a simulated first day of residency assessment. They were assessed twice with a questionnaire for communication and interpersonal skills (ComCare) by the simulated patients: directly after each consultation from the internal perspective of the patient’s view (internal perspective) and four month later from the 208 consultation videos from an external perspective of the patient’s view (external perspective). All eight ComCare items were assessed on a five-point Likert scale (1=full disagreement to 5=full agreement). Differences between the item means of internal and external perspective were examined by paired t-tests. Results: Overall, significantly higher ratings were found for all ComCare items from the external perspective except for the item “Interest”. Ratings for the items “Language” and “Listening” were significantly higher from the external perspective for all simulated patients. Significantly higher ratings for all items from the external perspective were observed for two simulated patients. Conclusion: Simulated patients’ ratings after a conversation seem to represent a more authentic view on students’ communication and interpersonal skills because of the emotionally experienced situation. The evaluation of those skills from a simulated patient perspective could be a valuable complement to communication ratings by examiners.


Introduction
Communication and interpersonals skills are important qualities of professionalism in medicine [1] and crucial elements in the delivery of good medical care and patient satisfaction [2], [3]. Undergraduate medical students usually receive communication trainings addressing empathy and interpersonal skills in their curriculum [4] and the assessment of communication skills often takes place in objective structured clinical examinations (OSCE) [5]. With respect to communication skills, mild, but significant correlations between simulated patients and examiners were found for warmth of greeting, listening skills, respect, and concern for the patient as a person in an OSCE [6]. While some studies found moderate to high correlations between simulated patients and examiners empathy ratings [7], [8], higher empathy ratings by simulated patients compared with examiner ratings in a clinical exam setting have been detected [9]. This study suggested further training for simulated patients to standardize empathy ratings and improve inter-rater reliability. Trained simulated patients have been shown to act as consistent raters for communications skills [10]. The research question for our study was, whether trained simulated patients assessed medical students' communication and interpersonal skills after physician-patient encounters from the internal perspective of the patient's view (short: internal perspective) in the same way than from an external perspective of the patient's view (short: external perspective) when rating the videographed interviews four months later. We hypothesize that a rating from these two different perspectives will lead to different assessments.

Project description and methods
Based on a validated 360-degree assessment for medical students, which simulates a first day of residency [11], we developed a competence-based training for final-year medical students in the newly founded Center for Training and Assessment of Medical Competences at the University Hospital Hamburg Eppendorf. In this training, participants complete three phases in the role of a physician: 1. a consultation hour with four simulated patients per participant, 2. a patient-management-phase with an online-patientdocumentation-form, and 3. a case presentation of one of the simulated patients.
In December 2019, 52 medical students in the first third of their final year participated in this training. The eight simulated patients -detailed role descriptions are described elsewhere [12] filled out a questionnaire for communication and interpersonal skills (ComCare) after every consultation [13]. Its eight items were assessed on a five-point Likert scale (1=full disagreement to 5=full agreement). All consultations were videographed (N=208). All simulated patients had received a rater training in December 2019. They rehearsed their respective patient role with either a real physician or a final year student who acted as their interview partners. After each interview, they filled out the ComCare questionnaire. The ComCare items were discussed and typical anchors to recognize behaviour linked to each item were defined. In April 2020, four month after the assessment, the same eight simulated patients watched their own consultation videos and filled out the ComCare again. All simulated patients received a written briefing before they rated the videos. They were told to rate the participants from the patient perspective, focussing on communicative and interpersonal aspects and not on medical or other issues. The time distance of four month before the second rating was chosen to give the actors additional opportunity to have achieved emotional distance from their own role. The videos were rated by the actors in the same chronological order and in groups of four videos similar to the rating during the original 360-degree assessment. Using SPSS Statistics 26 for statistical analysis, we calculated means and standard deviations for all ComCare items in total and per simulated patient. Additionally, we used paired t-tests to examine differences between both measurements (I: internal perspective, directly after each consultation, E: external perspective, four month later after watching their consultation videos). Due to multiple testing, p-Values were Bonferroni corrected. Cohen's d was calculated for effect sizes. Additionally, we calculated the mean differences between both ratings (internal and external perspective) and conducted an ANOVA with those for the items "language" and "listening", which showed significant differences for all simulated patients. For posthoc testing, we used Bonferroni corrections.

Results
Overall, simulated patients assessed all ComCare items significantly higher from the external perspective except for the item "Showing sincere interest" (see table 1). The greatest difference with large effect sizes between assessment from external versus internal perspective was found for the items "Language" (2.43 points, d=8.94) and "Listening" (1.68 points, d=3.97). Significantly higher ratings for these two items were also observed for all simulated patients individually from the external perspective (see table 2). For all items, significantly higher ratings were only observed for SP3: 46-year-old man with severe abdominal pain (abdominal migraine) and SP4: 45-yearold woman with dizziness and malaise (second degree atrioventricular block Mobitz type II). We found significant differences between the mean differences of both ratings for "language" and "listening" in all simulated patients   table 3). The greatest difference for "language" was found for SP6 (a talkative and cooperative patient) and for "listening" for SP7 (a friendly, thankful patient with intense pain). SP8 (a patient with fear of cancer) showed the lowest mean differences between the two ratings for "language" and SP1 (a taciturn and suborn patient) for "listening".

Discussion
Rating of medical students' interpersonal skills or empathy in communication situations with simulated patients can be carried out in different ways: by only examiners directly after the consultation [14], [15], by the simulated patients themselves and by examiners directly after the consultation [6], [7], [8], [9], or by examiners watching video recorded communication situations at a convenient time [16], [17]. In our study, simulated patients rated the same consultations from the patient perspective directly after the consultation and four months later from the external perspective of the patient's view, watching their videos. Interestingly, all simulated patients gave much better scores for "Use of plain language" and "Attentive listening" from the external perspective than from the internal, emotionally altered patient perspective. Real patients reported that they sometimes felt that doctor was not listening or used words they did not understand [18], [19]. Our simulated patients' lower ratings from the internal perspective could be a result of their perceived impression whereas they might have rated the video with emotional distance from the external perspective. Two simulated patients who played roles with very debilitating emotional states (SP3 is very angry and SP4 is stressed in her job), rated all aspects of the ComCare lower from the internal than from the external perspective. Regarding the aspect of "listening", the mean difference between the two ratings was, for example, significantly higher for a patient with intense pain (SP7) compared to a taciturn, stubborn patient (SP1).This gives another clue, that severe emotional alteration (e.g. by pain) can greatly hamper a (simulated) patient's evaluation of a (simulated) physician's communication skill. It has been shown before that ratings of communication skills are not independent of case content [20] and the acted emotions might have influenced the rating from the internal perspective in our setting versus the more distanced rating from the external perspective without the emotions of patient role. A strength of this study is that we deployed only one actor per patient role which supports high standardisation for role playing and rating. A limitation is that the actors could have gathered experiences between the two points of measurement, which could have influenced their rating at the second time point. Furthermore, we can only provide data from eight simulated patient roles, which limits the generalizability of our results. However, this pilot study provides first, interesting findings which can be explored further.

Conclusion
Simulated patient rating of communication and interpersonal skills after a conversation includes the internal patient perspective which is influenced by the patient's emotions and therefore seems to represent a more authentic patient view. We conclude, that the evaluation of communication and interpersonal skills from the emotionally involved simulated patient perspective can be a valuable complement to communication ratings by examiners. Students could benefit from the additional learning experience that feedback from an emotionally involved simulated patient can differ from the feedback they receive from an external perspective, which is mostly based on professional aspects of communication.  [14], [15], durch die simulierten Patient*innen selbst und durch Prüfer*innen direkt nach der Konsultation [6], [7], [8], [9] oder durch Prüfer*innen, die sich zu einem geeigneten Zeitpunkt videoaufgezeichnete Kommunikationssituationen ansehen [16], [17].